Code Coverage Analysis of Combinatorial Testing

Combinatorial t-way testing with small t is known as an e cient black-box testing technique to detect parameter interaction failures. So far, several empirical studies have reported the e↵ectiveness of t-way testing on fault detection abilities. However, few studies have investigated the e↵ectiveness of t-way testing on code coverage, which is one of the most important coverage criteria widely used for software testing. This paper presents a quantitative analysis to evaluate the code-coverage e↵ectiveness of t-way testing. Using three open source utility programs, we compare t-way testing with exhaustive (all combination) testing w. r. t. code coverage and test suite sizes. Keywords-Combinatorial testing; t-way testing; Exhaustive testing; Code coverage; Line coverage; Branch coverage.